The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, the approaches described in this section are not prior art to the claims in this application and are not admitted to be prior art by inclusion in this section
Applications that are used with computing systems can be enhanced or extended by using plug-in applications, also commonly referred to simply as “plug-ins.” As used herein, the term “plug-in” refers to an auxiliary application that works with another application to enhance one or more features or capabilities of the other application. For example, plug-ins are often used with Web browser applications to provide support for additional types of Web page content, such as audio, video, etc. A plug-in is typically referenced by a special hypertext markup language (HTML) tag on a Web page, and when the Web browser loads the Web page, the browser retrieves and executes the code for the plug-in. Plug-ins often refer to auxiliary applications in software, but may also refer to auxiliary applications implemented in hardware or implemented through a combination of hardware and software.
Applications can be enhanced or extended by using applet applications, also commonly referred to simply as “applets.” As used herein, the term “applet” refers to an application that has limited capabilities as compared to typical applications. For example, an applet may contain one or a few number of features, require limited memory, be limited in size, be portable among different operating systems, or may include one or more of such characteristics. Applets are commonly distributed with Web pages and are executed using Web browsers. Applets are typically executed by a plug-in that operates in conjunction with the Web browser.
Examples of applets include Java applets that are based on the Java programming language. An example of a plug-in is the Java Plug-in that is provided by Sun Microsystems. The Java Plug-in allows applets included in Web pages to be executed using Sun's Java 2 Runtime Environment, Standard Edition (J2RE) instead of the default Java virtual machine (JVM) that is included in a Web browser. Examples of Web browsers include Netscape Navigator and Microsoft Internet Explorer. The use of the Java Plug-in allows users of Web browsers to use a more current version of the JVM than the default JVM included in the user's particular version of their Web browser because a newer version of the Java Plug-in can be obtained and installed more easily than obtaining and installing a new Web browser version.
The applets and plug-ins that are executed with an application are generally independent of the “operating environment” used to execute the applets and plug-ins. As used herein, the term “operating environment” refers to one or more parameters that can affect the execution of the applets, plug-ins, and applications, including but not limited to, one or more of the following: the particular application, the particular version of the application, the operating system, and the hardware platform that executes the particular application. For example, the operating environment may include version 2.0 of the Netscape Navigator Web browser running on an Intel-based computer that uses the Microsoft Windows XP operating system.
Although applets and plug-ins are designed to be independent of the operating environment, the differences in applications, application versions, operating systems, and hardware platforms may lead to differences when executing applets and plug-ins in different operating environments. As a result, developers of applets and plug-ins test their products in many different operating environments. Similarly, developers of applications test their products to ensure that the applications correctly execute different applets and plug-ins in different operating environments.
One problem with such testing of applets, plug-ins, and applications in different operating environments is the large number of potential tests based on the many different combinations of operating environment parameters, such as the number of different applications, application versions, operating systems, hardware platforms, applets, applet versions, plug-ins, and plug-in versions. As a result of so many different parameters, the number of different operating environments are often at least in the dozens, if not hundreds. Although not all operating environments are always supported, the supported number can still be considerable, and therefore the testing effort is significant.
One approach for testing applications in different operating environments is to execute a number of test cases in each operating environment. For example, to test a set of five applets with a particular version of the Java Plug-in, a user can use a Web page that includes a link to each of the five applets. By clicking on each link to each applet, the user can execute the set of applets, thereby testing whether the applets are successfully executed by the Java Plug-in.
However, there are several disadvantages to this approach. For example, the user manually executes each applet by clicking on each link to each applet used in testing. For tests that involve a large number of applets, the time required for a user to click on each link to each applet can be considerable. Furthermore, the time required for testing rapidly increases as the number of operating environments increases. It is not uncommon for a user to need several days to execute a full testing sequence for a set of operating environments. Finally, the user may make mistakes in running a series of test cases, resulting in some cases not being executed, other cases being executed multiple times, or some cases being improperly executed. This can be a problem when the same test cases are used later to test another set of conditions (e.g., to test another plug-in or application) because the test cases may not be run the second time in the same way as the first time. This can lead to differences in the test results that are due to the testing process, not the things being tested.
Another problem with this approach is that applets often work “behind the scenes” with the application. For example, applets that run with a plug-in for a Web browser may add capabilities to the Web browser that are not visible to the user. As a result, the user may be unable to determine whether or not the applet was successfully executed.
Based on the foregoing, there exists a need for a mechanism for testing execution of applets with plug-ins and applications that minimizes the time required for the testing. There also exists a need for a mechanism for testing execution of applets with plug-ins and applications that allows for a determination to be made whether the applet was successfully executed with the plug-in and application when the effect of executing the applet is not visible to a user.